12 research outputs found
Multimodal Classification of Parkinson's Disease in Home Environments with Resiliency to Missing Modalities
Parkinson’s disease (PD) is a chronic neurodegenerative condition that affects a patient’s everyday life. Authors have proposed that a machine learning and sensor-based approach that continuously monitors patients in naturalistic settings can provide constant evaluation of PD and objectively analyse its progression. In this paper, we make progress toward such PD evaluation by presenting a multimodal deep learning approach for discriminating between people with PD and without PD. Specifically, our proposed architecture, named MCPD-Net, uses two data modalities, acquired from vision and accelerometer sensors in a home environment to train variational autoencoder (VAE) models. These are modality-specific VAEs that predict effective representations of human movements to be fused and given to a classification module. During our end-to-end training, we minimise the difference between the latent spaces corresponding to the two data modalities. This makes our method capable of dealing with missing modalities during inference. We show that our proposed multimodal method outperforms unimodal and other multimodal approaches by an average increase in F1-score of 0.25 and 0.09, respectively, on a data set with real patients. We also show that our method still outperforms other approaches by an average increase in F1-score of 0.17 when a modality is missing during inference, demonstrating the benefit of training on multiple modalities
Protocol for PD SENSORS:Parkinson’s Disease Symptom Evaluation in a Naturalistic Setting producing Outcomes measuRes using SPHERE technology. An observational feasibility study of multi-modal multi-sensor technology to measure symptoms and activities of daily living in Parkinson’s disease
Introduction The impact of disease-modifying agents on disease progression in Parkinson’s disease is largely assessed in clinical trials using clinical rating scales. These scales have drawbacks in terms of their ability to capture the fluctuating nature of symptoms while living in a naturalistic environment. The SPHERE (Sensor Platform for HEalthcare in a Residential Environment) project has designed a multi-sensor platform with multimodal devices designed to allow continuous, relatively inexpensive, unobtrusive sensing of motor, non-motor and activities of daily living metrics in a home or a home-like environment. The aim of this study is to evaluate how the SPHERE technology can measure aspects of Parkinson’s disease.Methods and analysis This is a small-scale feasibility and acceptability study during which 12 pairs of participants (comprising a person with Parkinson’s and a healthy control participant) will stay and live freely for 5 days in a home-like environment embedded with SPHERE technology including environmental, appliance monitoring, wrist-worn accelerometry and camera sensors. These data will be collected alongside clinical rating scales, participant diary entries and expert clinician annotations of colour video images. Machine learning will be used to look for a signal to discriminate between Parkinson’s disease and control, and between Parkinson’s disease symptoms ‘on’ and ‘off’ medications. Additional outcome measures including bradykinesia, activity level, sleep parameters and some activities of daily living will be explored. Acceptability of the technology will be evaluated qualitatively using semi-structured interviews.Ethics and dissemination Ethical approval has been given to commence this study; the results will be disseminated as widely as appropriate
Weakly-Supervised Completion Moment Detection using Temporal Attention
Monitoring the progression of an action towards completion offers fine
grained insight into the actor's behaviour. In this work, we target detecting
the completion moment of actions, that is the moment when the action's goal has
been successfully accomplished. This has potential applications from
surveillance to assistive living and human-robot interactions. Previous effort
required human annotations of the completion moment for training (i.e. full
supervision). In this work, we present an approach for moment detection from
weak video-level labels. Given both complete and incomplete sequences, of the
same action, we learn temporal attention, along with accumulated completion
prediction from all frames in the sequence. We also demonstrate how the
approach can be used when completion moment supervision is available. We
evaluate and compare our approach on actions from three datasets, namely HMDB,
UCF101 and RGBD-AC, and show that temporal attention improves detection in both
weakly-supervised and fully-supervised settings
Action Completion: A Temporal Model for Moment Detection
We introduce completion moment detection for actions - the problem of
locating the moment of completion, when the action's goal is confidently
considered achieved. The paper proposes a joint classification-regression
recurrent model that predicts completion from a given frame, and then
integrates frame-level contributions to detect sequence-level completion
moment. We introduce a recurrent voting node that predicts the frame's relative
position of the completion moment by either classification or regression. The
method is also capable of detecting incompletion. For example, the method is
capable of detecting a missed ball-catch, as well as the moment at which the
ball is safely caught. We test the method on 16 actions from three public
datasets, covering sports as well as daily actions. Results show that when
combining contributions from frames prior to the completion moment as well as
frames post completion, the completion moment is detected within one second in
89% of all tested sequences